Papers with document understanding benchmarks
LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating (2025.acl-long)
Copied to clipboard
Chao Deng, Jiale Yuan, Pi Bu, Peijie Wang, Zhong-Zhi Li, Jian Xu, Xiao-Hui Li, Yuan Gao, Jun Song, Bo Zheng, Cheng-Lin Liu
| Challenge: | Existing document understanding benchmarks only handle a small number of pages . existing models are limited to handling only a limited number of documents . |
| Approach: | They propose a long document understanding benchmark that integrates three primary tasks and 20 sub-tasks based on different primary tasks. |
| Outcome: | The proposed model outperforms existing benchmarks on open-source and closed-source models . the model outpersforms other models on more than 33,000 pages of documents . |
ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images (2026.eacl-long)
Copied to clipboard
Mathieu Sibue, Andrés Muñoz Garza, Samuel Mensah, Pranav Shetty, Zhiqiang Ma, Xiaomo Liu, Manuela Veloso
| Challenge: | Existing models for structured information extraction are limited by narrow entity ontologies, simple queries, or homogeneous document types. |
| Approach: | They propose a benchmark dataset for structured Information Extraction (IE) from document images . they analyze open and closed VLMs on this benchmark . |
| Outcome: | The proposed model can perform fine-grained structured extraction across document types and schemas. |